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Abstract 


This  thesis  examined  techniques  used  to  derive 
estimates  of  the  cost  at  completion  for  various  research  and 
development  programs.  The  six  methods  examined  were  the 
methods  used  in  the  Cost  Performance  Report  Analysis  (CPRA) 
computer  program  to  calculate  estimates  at  completion. 

The  analysis  is  based  on  a  linear  regression  between 
the  cost  at  completion  and  the  estimate  at  completion  for 
each  technique  available.  The  techniques  were  ranked  by 
coefficient  of  determination  and  a  general  linear  test  was 
performed  to  test  for  equality  among  the  regression  lines. 

The  results  of  this  investigation  indicate  that  an 
estimate  at  completion  based  upon  weighted  cost  and  schedule 
indices  minimizes  the  unexplained  error  (as  a  percentage  of 
total  error)  and  is  thought  to  be  the  superior  forecaster  of 
costs  at  completion.  The  general  linear  test  for  equality 
among  the  regression  lines  generated  by  the  different 
techniques  did  not  indicate  the  existence  of  commonality 
between  regression  lines.  This  means  that  each  technique 
tested  provided  a  unique  estimate  at  completion. 
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AN  EVALUATION  OF  CPRA  ESTIMATE  AT  COMPLETION  TECHNIQUES 
BASED  UPON  AFWAL  COST/SCHEDULE  CONTROL  SYSTEM  CRITERIA 

DATA 


I.  Introduction 


General  Issue 


The  Department  of  Defense  (DOD)  implemented  the 
Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  as  a  "tool 
for  use  in  assessing  a  contractor's  cost  management  system 
circa  1967"  (1:32).  "The  objective  of  DOD  in  applying  C/SCSC 
to  a  contract  is  to  obtain  assurance  that  the  management 
system  of  a  contractor  has  the  capability  to  adequately  plan 
and  measure  actual  contract  performance  and  that  the 
management  system  can  provide  financial  and  related  progress 
reports  from  a  common  auditable  data  base"  (2:12). 

Therefore,  as  a  minimum  a  contractor's  management  control 
system  must  provide  for: 

A.  Realistic  budgets  for  work  scheduled  within 
responsibility  assignments  (3:5). 

B.  Accurate  accumulation  of  costs  related  to 
progress  of  the  planned  work  (3:5,  , 

C.  Comparison  between  the  actual  resources 
applied  and  the  estimated  resources  planned  for 
specific  work  assignments  (3:5). 

D.  Preparation  of  reliable  estimates  of  costs 
to  complete  remaining  work  (3:5). 

E.  Support  of  an  overall  capability  for  managers 
to  analyse  available  information  to  identify 
problem  areas  in  sufficient  time  to  take  remedial 
action  (3:5). 
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While  management  systems  adhering  to  this  criteria  present 
cost  and  schedule  data  in  a  standardized  manner,  "the  great 
potential  for  better  cost  and  schedule  control  is  lost  if  the 
data  is  not  analyzed  and  made  to  produce  essential  management 
information  needed  by  the  Program  Manager  and  senior 
managers"  (4:273).  Indeed,  if  this  data  is  analyzed 
properly,  it  can  provide  management  with  both  a  historical 
and  projected  cost  perspective.  The  projected  cost,  called 
an  Estimate  At  Completion  (EAC),  is  one  of  the  analyst's  key 
measures  of  the  contractor's  performance  (4:277). 

In  fact,  "the  contract  status  information  on  the  [Cost 
Performance  Report)  provides  a  basis  for  verifying  the 
contractor's  estimate  of  cost  at  completion,  or  for 
developing  an  independent  estimate  thereof"  (4:277).  "Early 
visibility  of  cost  and  schedule  problems  must  result  in  the 
reassessment  of  the  ultimate  cost  and  timely  changes  to 
program  budgets  and  fiscal  plans"  (4:277).  For  this  reason, 
the  EAC  is  one  of  the  primary  items  needed  by  the  government 
to  make  effective  management  decisions  (2:73).  Therefore,  it 
is  important  that  the  analyst  understand  the  techniques  and 
methods  for  developing  EACs  and  their  limitations. 

Specific  Problem 

Six  methods  are  used  to  develop  EACs  at  the  Air  Force 
Wright  Aeronautical  Laboratories  (AFWAL)  at  Wright-Patterson 
Air  Force  Base  (WPAFB),  Ohio.  Currently,  the  method  employed 
to  develop  an  EAC  is  selected  by  the  analyst  based  on  that 


2 


analyst ' s  experience  ( 5 ) .  In  some  instances  analysts 
will  generate  six  EACs  using  the  Cost  Performance  Report 
Analysis  (CPRA)  computer  program,  and  use  their  arithmetic 
average  as  the  "true"  EAC  (5),  The  problem  with  averaging 
the  six  EACs  is  that  if  the  actual  program  cost  at  completion 
differs  substantially  from  one  or  more  of  the  six  EACs,  that 
EACs  inclusion  in  the  averaged  EAC  will  actually  increase 
the  variance  the  analyst  wishes  to  minimize  (6:72-73).  If 
the  past  performance  of  one  EAC  technique  had  been  shown  to 
outperform  the  others  under  given  conditions ,  the  analyst 
could  rule  out  those  EAC(s)  more  likely  to  degrade  the 
quality  of  the  final  prediction.  The  desirability  of 
understanding  the  available  techniques  and  their  application 
was  noted  in  Holeman’s  1974  essay;  "there  does  not  appear  to 
have  been  any  attempt  to  conduct  in-depth  or  independent 
reviews  of  any  of  these  automated  approaches  to  offer 
unbiased  and  expert  guidance  on  when  and  where  they  should  or 
should  not  be  used"  (7:40).  Holeman  adds,  "these  techniques 
.  .  .  could  be  considered  analytically  sophisticated  and  many 
analysts  in  program  offices  would  probably  have  difficulty  in 
fully  understanding  the  supporting  documentation"  (7:40-41). 
As  implied  above,  this  lack  of  understanding  may  cause  the 
analyst  to  misestimate  completion  costs.  This  thesis 
attempts  to  evaluate  the  EAC  techniques  used  by  the  AFWAL's 
automated  analysis  program  for  C/fcCS  generated  data. 
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Background 


"C/SCSC  resulted  from  the  realization  that  a  contractor 
receives  payment  not  only  for  a  defense  system  development 
and  production  program,  but  also  for  management"  (8:43). 
Government  personnel  knew  that  "to  achieve  reliable 
visibility  of  the  contractor's  adequacy  of  planning,  the 
government  must  use  the  same  data  the  contractor  used  in 
preparing  his  plan"  (7:37).  Early  attempts  to  access  this 
data  were  implemented  by  the  Air  Porce's  Cost/Schedule 
Planning  and  Control  Specifications,  C/SPCS"  (2:3,8:43). 

"This  criteria  approach  was  monitored  closely  by  the  other 
military  services  ..no  gradually  evolved  into  the  present 
C/SCSC"  (2:3).  "'Criteria'  is  the  key  word  in  C/SCSC" 

(9:32).  Simply  put,  "it  is  a  criteria  or  a  set  of  standards 
that  a  contractor's  management  system,  whatever  it  may  be, 
must  meet  in  undertaking  development  of  a  major  defense 
system"  (9:32).  "Surprising  to  most  contractors  since  that 
time  (validation  of  the  first  contractor's  management  system 1 
has  been  the  fact  that  a  good  existing  system  is  generally 
90%  satisfactory  insofar  as  the  criteria  are  concerned" 
(4:263-4).  Much  of  C/SCSC 's  success  has  been  attributed  to 
the  fact  that  it  is  not  a  system,  but  a  set  of  criteria  and 
therefore  does  not  mandate  a  particular  managerial  style  or 
technique  (9:33). 

"Essentially  what  C/SCSC  does  is  ensure  that  data 
provided  by  a  contractor,  such  as  the  monthly  Cost 
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Performance  Report  (CPR)  [or  the  Cost /Schedule  Status  Report 
(C/SSR) ,  depending  on  the  dollar  value  of  the  contract],  is 
accurate  and  timely"  (9:32-33).  At  a  minimum,  contractor's 
management  and  control  systems  should  provide  for  "support  of 
an  overall  capability  for  managers  to  analyze  available 
information  to  identify  problems  areas  in  sufficient  time  to 
take  remedial  action"  (3:5).  One  of  the  major  concepts  that 
C/SCSC  addresses  is  that  of  accurately  measuring  and  relating 
incurred  costs  and  budgeted  costs  (3:5).  In  other  words , 
C/SCSC  requires  that  the  contractor  maintain  the  capability 
to  compare  the  Actual  Cost  of  Work  Performed  (ACWP)  with  the 
Budgeted  Cost  of  Work  Performed  (BCWP),  and  the  BCWP  with  the 
Budgeted  Cost  of  Work  Scheduled  (BCWS)  (3:5).  These  terms, 
when  mathematically  related  to  each  other,  are  the  basis  for 
EAC  formulation  (10,11:17-19).  A  complete  discussion  of 
C/SCSC  can  be  found  in  Air  Force  Systems  Command  Pamphlet 
( AFSCP  173-5)  titled  Cost  Analysisf:!  Cost/Schedule  Control 
Systems  Criteria  Joint  Implementation  Guide  ( 3 ) .  A  subset  of 
common  C/SCSC  terms  are  defined  in  the  next  section. 

C/SCSC  Terms  and  Definitions.  The  following  terms 
are  used  throughout  this  section  and  are  defined  for  the 
convenience  of  the  reader. 

A.  Work  Package  -  Detailed  short  span  jobs, 
or  material  items,  identified  by  the  contractor 
for  accomplishing  work  required  to  complete  the 
contract  (3:6). 
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B.  Actual  Cost  of  Work  Performed  (ACWP)  -  The 
costs  actually  incurred  and  recorded  in 
accomplishing  the  work  performed  within  a  given 
time  period  (3:5). 


C.  Budgeted  Cost  for  Work  Performed  (BCWP)  - 
The  sum  of  the  budgets  for  completed  work 
packages  and  completed  portions  of  open  work 
packages,  plus  the  appropriate  portion  of  the 
budgets  for  level  of  effort  and  apportioned 
effort  (3:6). 


D.  Budgeted  Cost  for  Work  Scheduled  (BCWS)  - 
The  sum  of  the  budgets  for  all  work  packages 
scheduled  to  be  completed,  plus  the  budget  for 
the  portion  of  in-rroces&  work  (open  work 
packages)  tchtiuled  to  be  accomplished,  plus  the 
budgets  for  LOE  and  apportioned  effort  scheduled 
to  be  completed  during  the  period  (12). 


E.  Latest  Revised  Estimate  (LRF)  -  The  latest 
revised  estimate  of  contract  costs  at  the 
completion  of  the  contracted  effort.  Generally, 
lre  and  EAC  are  synonymous  acronyms j  in  this 
paper  the  convention  ot  using  LRE  to  refer  to  a 
contractor  estimate  and  EAC  to  refer  to  a 
government  generated  estimate  will  be  employed. 


F.  Budget  at  Completion  (BAC)  -  The  budgetary 
goal,  excluding  management  reserve,  for  doing  all 
the  authorized  work  (13:16).  The  BAC  is  the 
endpoint  of  the  Performance  Measurement  Baseline 
(13:17) . 


G.  Performance  Measurement  Baseline  (PMB)  - 
A  graphical  depiction  of  the  BAc  spread  over 
the  time  allotted  for  performance.  A  time-phased 
budget  plan.  The  progressive  accumulation  of  the 
BCWS  to  be  accomplished  in  each  increment  of  time 
throughout  tne  contract  period  of  performance 
(13:17). 


Cost  Performance  Report  Analysis  (CPRA)  Program. 


CPRA  program  is  a  computer  program  which,  based  upon 


contractor  performance  measurement  data  discussed  earlier, 
performs  the  calculations  necessary  to  develop  EACs.  The 
program  is  written  in  FORTRAN  IV  and  resides  on  the 
Aeronautical  Systems  Division's  Cyber  computer.  Although  the 
CPRA  program  provides  the  AFWAL  analyst  with  many  different 
types  of  analyses,  the  EACs  that  are  generated  are  the  focus 
of  this  thesis. 

The  program  uses  three  constructs  to  generate  EACs. 
These  constructs  are  based  on  a  cost  index,  a  schedule  index 
and  trend  analysis.  These  methods  are  explained  in  detail  in 
the  "EAC  Formulas"  section  of  this  report.  The  EAC 
techniques  used  by  the  CPRA  program  were  duplicated  on  the 
Zenith  Z-1QQ  microcomputer.  This  was  done  because  of  the 
expense  associated  with  using  a  mainframe  (large)  computer  to 
accomplish  a  task  capably  performed  by  a  microcomputer 
( small ) .  The  results  obtained  from  the  CPRA  simulation  were 
fed  into  a  computer  based*  statistics  program,  the 
Statistical  Package  for  Social  Sciences**  (SPSS)  program, 
which  performed  the  majority  of  the  statistical  calculations 
described  in  the  "Methodology"  section  of  this  report.  Many 

*  Devore  notes  that,  "Whenever  possible  it  is  preferable  to 
do  regression  analysis  using  a  standard  statistical  computer 
program  package;  in  addition  to  3q  and  0},  the  resulting 
output  will  yield  much  more  useful  information"  (14:429). 

**  SPSS  is  a  well  known,  well  documented  statistical  package 
which  began  its  development  in  1965  at  Stanford  University 
<15:xxii).  This  statistical  package  is  now  being  used  at 
nearly  500  installations,  including  conversions  to  almost  20 
different  operating  systems  and  computers  (15:xxi). 


of  the  techniques  used  in  the  CPRA  program  are  common 
knowledge  among  C/SCS  analysts.  In  fact,  some  of  the  j 
techniques  may  be  found  in  "Estimates  at  Completion" 
(10:5-18),  a  pamphlet  distributed  in  the  SYS  362, 
Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  course  at  the 
Air  Force  Institute  of  Technology,  Wright  Patterson  AFB, 

Ohio. 

Description  of  CPRA  Terms  and  Formulas.  The  following 
information  is  provided  to  help  the  reader  understand  the 
mechanics  of  the  CPRA  program  for  calculating  EACs  and  has 
been  reproduced  from  the  CPRA  Users  Manual.  In  some 
equations  it  is  necessary  to  reference  two  similar 
performance  measurement  data  points  (for  example,  two 
different  ACWP  measurements)  in  one  equation;  in  this 
instance  the  following  convention  will  be  used.  The 
measurement  will  retain  its  usual  label,  for  example  "ACWP", 
and  will  also  have  an  array  position  indicator  added.  The 
indicator  "(CUM)"  will  be  added  to  the  label  to  indicate  the 
latest  month *s  cumulative  data  point.  For  example,  if 
our  last  data  point  were  June  198X,  the  actual  cost  of 
work  performed  from  the  start  of  the  contract  through 
the  month  of  June  would  be  represented  by  "ACWP(CUM)". 

The  array  position  indicator  for  the  preceding  month, 

May,  would  be  determined  by  its  relationship  to  the  current 
month.  In  other  words,  since  May  is  one  month  before  June, 
its  indicator  would  be  (CUW-1)  and  the  cumulative  ACWP  for 


8 


May  would  be  represented  as  "ACWP(CUM-l)".  "(CUM)"  will 
always  refer  to  the  latest  month’s  cumulative  data  and  all 
other  points  will  take  the  general  form  "(CUM-n);  where  "n" 
is  the  relative  number  of  months  before  the  latest  month.  If 
there  is  no  "(CUM)"  indicator  in  an  equation,  the  latest 
month's  cumulative  data  is  being  used. 


A.  Cost  Variance  %  =  [(BCWP  -  ACWP)  /  BCWP] 
x  100 

Positive  value  indicates  work  performed 
is  costing  less  than  planned  by  this  percentage. 
Negative  value  indicates  work  performed  is 
costing  more  than  planned  by  this  percentage 
(11:12). 


Current  Month  CPI  =  BCWP (CUM)  -  BCWP(CUM-l) 

ACWP (CUM)  -  ACWP ( CUM- 1) 


This  index  is  a  measure  of  cost  performance 
for  the  latest  month's  work  (11:12). 


C.  3  Month  CPI  =  BCWP (CUM)  -  BCWP (CUM- 3) 

ACWP (CUM )  ^  ACWP( CUM- 3 ) 

This  index  is  a  measure  of  cost  performance 
over  the  last  three  months  (11:12). 


D.  Cum  CPI  =  BCWP ( CUM )  +  ACWP(CUM) 

This  index  is  a  measure  of  cost  performance 
from  beginning  of  contract  through  the  our rent 
month  (11:14). 


E.  Schedule  Variance  %  »  ((BCWP-BCWS)  /  SCWS]  x 
100 

Positive  value  indicates  more  work  than 
planned  has  been  performed  by  this  percentage. 
Negative  value  indicates  less  work  than  planned 
has  been  performed  by  this  percentage  (12). 
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F.  Current  Month  SPI  =  BCWP(CUM)  -  BCWP(CUM-l) 

BCWS(CUM).  -  BCWS(CUM-l) 

This  index  is  a  measure  of  schedule  efficiency 
for  work  performed  this  month.  It  indicates  the 
efficiency  with  which  the  contractor  has 
accomplished  the  planned  amount  of  work  ( 12 ) . 


G.  3  Month  SPI  -  BCWP(CUM)  -  BCWP(CUM-3) 

BCWS(CUM)  -  BCWS( CUM-3 ) 

This  index  is  a  measure  of  schedule 
efficiency  for  work  performed  in  the  last  3 
months  (12). 


EAC  Formulas.  The  formulas  used  in  the  CPRA  program  to 
estimate  the  cost  of  a  program  at  completion  have  been 
reproduced  from  the  CPRA  Users  Manual  and  are  shown  below. 

The  EAC  techniques  have  been  numbered  1  through  6  and  are 
referenced  by  the  acronym  EAC  with  a  reference  number,  "n", 
appended  to  the  ,,EAC,,  stem.  For  example  the  first  technique 
developed  is  based  on  the  current  month  CPI  and  is  referred 
to  as  EAC1,  the  second  technique  is  referred  to  as  EAC2  and 
so  on.  "EACn",  where  Mn"  is  a  reference  number  from  1  to  6, 
will  be  used  to  describe  a  particular  technique  throughout 
the  remainder  of  this  thesis. 

A.  Based  on  Current  Month  CPI: 

EACl  »  ACWP(CUM)  +  BAC(CUM)  -  BCWP(CUM) 

CUR  MON  CPI 

This  calculation  assumes  the  remaining  work 
will  be  accomplished  at  the  same  budget 
efficiency  (CPI)  as  that  exhibited  last  month 
(11:17). 
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B.  Based  on  Cumulative  CPI  (11:17): 

EAC2  =  ACWP(CUM)  +  BAC(CUM)  -BCWP(CUM) 

CUM  CPI 

This  calculation  assumes  remaining  work 
will  be  accomplished  at  the  same  budget 
efficiency  as  has  been  exhibited  cumulatively 
since  the  beginning  of  the  contract  (12). 


C.  Based  on  Three  Month  CPI  (11:17): 

EAC3  =  ACWP(CUM)  t  BAC(CUM)  -  BCWP(CUM) 

THREE  MON  CPI 

This  calculation  assumes  the  remaining  work 
will  be  accomplished  at  the  same  budget 
efficiency  as  was  exhibited  cumulatively  during 
the  last  three  months  (12). 


D.  Based  on  ACWP  Regression:  This  calculation 
uses  a  least-squares-best-fit  on  ACWP  to 
establish  a  trend  line.  This  trend  line  is  then 
hypothetically  extended  to  the  point  in  time 
when  there  is  zero  work  remaining.  The 
hypothetical  ACWP  at  this  point  is  EAC4  (12). 


E.  Based  on  Weighted  SPI/CPI: 

ETC  a  [100.  -  (COST  VAR  %)  +  .75  X 

(SCHEDULE  VAR  %)]  X  (BCWR) 

100 


EAC5  =  ETC  +  ACWP 

This  method  calculates  EAC5  based  on 
one-time  completion  of  the  program.  This 
calculation  is  only  made  after  there  is  [sic]  six 
months  of  data  for  the  line  item.  The 
calculation  assumes  the  Cost  Variance  %  will 
remain  constant  to  the  end  of  the  program.  A 


Logically,  it  would  appear  that  a  program  may  be  completed 
just  once,  therefore  the  author  has  assumed  that  "one-time 
completion"  actually  means  "on-time  completion”  or  more 
appropriately  "on  schedule  completion" , 
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penalty  of  3/4  of  [the]  schedule  variance  is  used 
to  calculate  catch-up  (11:18). 


F.  Based  on  Trend  Weighted  SPI/CPI: 

EACC  =  (.12  x  EAC1)  +  (.24  x  EAC3)  +  (.64  x  EAC2) 

Cost  to  catch  up  schedule  is  calculated  by  the 
following  (11:18): 

ETCS  =  (MONTHS  BEHIND  SCHEDULE)  x  ACWP  RATE*  x  .75 
These  are  then  added  together  to  form  EAC6  (11:18): 
EAC6  =  EACC  +  ETCS 

This  prediction  assumes  a  schedule  catch-up 
penalty  (based  on  75%  of  the  spend  rate) and  a 
budget  probability  based  on  the  current  month 
cost  performance  index,  last  three  months  cost 
performance  index,  and  cumulative  cost 
performance  index  EAC  weighted  at  12%,  24%  and 
64%  respectively  (12). 


Scope  and  Limitations 

The  information  presented  as  background  material 
in  this  chapter  is  the  result  of  a  literature  review  of 
material  gathered  from  the  following  sources:  The  Defense 
Technical  Information  Center  (DTIC);  The  Electronic  Systems 
Division  (ESD)  Cost  Library  at  Hanscom  Air  Force  Base, 
Massachusetts;  The  Aeronautical  Systems  Division  (ASD)  Cost 
Library  at  Wright  Patterson  Air  Force  Base  ( WPAFB )  Ohio;  and 
The  Air  Force  Institute  of  Technology  Library,  also  located 
at  WPAFB. 

*  ACWP  RATE  is  defined  in  the  body  of  the  CPRA  Program  as 
ACWP (CUM)  /  TNUM  where  TNUM  is  the  number  of  months  from 
contract  start  to  the  last  month  for  which  data  is  available. 
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This  research  effort-  conducted  a  statistical  evaluation 
of  the  forecasting  effectiveness  of  each  EAC  technique  used 
by  the  CPRA  program.  It  should  be  noted  that  all  EAC's  which 
result  from  the  CPRA  program  (and  CPR/CSSR  analysis  in 
general)  rely  on  past  performance  to  predict  future 
performance.  Therefore,  the  ability  of  the  CPRA  program  to 
accurately  predict  variations  in  a  program  which,  to  date, 
has  experienced  no  significant  variation  is  limited. 
Additionally  it  should  be  noted  that  engineering  change 
proposals,  contract  stretches  due  to  ’’external ’’  influences 
(congressional,  labor  strike,  etc.)  are  facts  of  life 
in  the  acquisition  environment  and  may  have  a  confounding 
impact  on  the  ability  to  measure  the  effectiveness  of 
the  CPRA  forecasting  techniques.  The  data  contained  in 
the  AFWAL  CPR  and  C/SSR  database  and  the  EACs  generated 
from  that  database  using  the  CPRA  program  techniques  were  the 
inputs  to  this  analysis  and  are,  therefore,  subject  to  the 
aforementioned  anomalies . 

Research  Objectives. 

The  research  hypothesis  is  that  statistical  evaluation 
of  the  past  performance  of  each  method  used  to  derive  EACs 
may  indicate  a  preferred  method  of  formulation.  The 
objective  of  this  research  was  to  identify  this  preferred 
method,  if  one  exists. 

Investigative  Questions.  The  following  investigative 
questions  will  be  answered. 
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1.  Are  the  lines  formed  by  a  plot  of  the  CPRA 
program  EACs  and  estimated  cost  at  completion 
linear? 

If  present,  linearity  between  the  actual  cost  at  completion 
and  the  estimate  at  completion  permits  the  use  of  regression 
analysis  for  evaluating  the  CPRA  EAC  techniques. 

2.  Which  regression  line  has  the  highest 
coefficient  of  determination? 

Since  the  coefficient  of  determination  is  a  regression 
model's  ratio  of  explained  variation  to  its  total  variation 
(14:455),  the  regression  model  which  has  the  highest 
coefficient  of  determination  will  minimize  the  unexplained 
variation.  In  this  thesis,  the  technique  with  the  greatest 
coefficient  of  determination  is  defined  as  the  "best 
estimator" , 

3.  Are  there  significant  differences  between 
the  regression  lines  used  to  estimate  the  EACs? 

If  a  general  linear  test  shows  that  a  pair  of  regression 
lines  are  statistically  equivalent,  the  techniques  used  to 
generate  them  are  also  equivalent.  This  test,  therefore, 
enables  the  analyst  to  determine  whether  2  EACs  with 
different  formulas  for  calculation  are,  in  reality,  just 
different  ways  of  expressing  a  single  EAC. 
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II,  Methodology 


The  Database 

The  database  used  in  this  research  was  obtained  from 
the  Air  Force  Wright  Aeronautical  Laboratory  ( AFWAL) .  The 
database  is  comprised  of  performance  measurement  data 
from  57  on-going  research  and  development  programs  and 
contains  more  than  800  data  records*.  The  following 
programmatic  data  was  available  for  each  of  the  57  programs: 
contract  name  and  number;  current  and  original  target  cost; 
contract  start  and  stop  date  and  contract  type.  Each  data 
record  contained  the  following  elements;  BCWS,  BCWP,  ACWP,  MR 
(management  reserve),  BAG,  LRE  and  DATE  (month  and  year  of 
data  observation) .  The  database  was  reformatted  to  include 
the  programmatic  data  in  each  data  record.  Program  numbers 
were  assigned  sequentially  to  each  program.  These  numbers 
were  used  to  facilitate  the  referencing  of  individual 
programs  when  performing  calculations. 

EAC  Generation.  The  contractor  performance  measurement 
data  described  in  the  previous  paragraph  was  used  as  input 
for  a  FORTRAN  computer  program  which  generated  EACs  using 
techniques  from  the  CPRA  program. 

The  CPRA  program  does  not  calculate  EACS  or  EAC6  until 
eight  months  of  program  data  have  been  accumulated.  The  CPRA 

*  A  data  record  is  comprised  of  all  the  data  elements 
(individual  descriptors)  needed  to  completely  define  a  single 
month's  performance  measurement  data  set. 
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User's  Manual  offers  no  explanation  of  the  eight  month 
minimum  data  requirement  and  a  review  of  the  C/SCSC 
literature  did  not  validate  this  seemingly  arbitrary  decision 
rule.  Therefore,  this  restriction  on  the  application  of 
these  techniques  has  not  been  imposed  when  formulating  the 

i 

EAC  database  used  in  this  thesis. 

The  CPRA  program  assigns  the  value  of  .1  to  data 
points  which  otherwise  would  result  in  division  by  zero. 

This  practice  also  appeared  to  be  arbitrary  in  nature  and  was 
not  adopted.  For  example,  when  determining  the  current 
month's  CPI,  (BCWP  /  ACWP),  ACWP ( CUM- 1)  is  subtracted  from 
ACWP(CUM)  to  calculate  a  single  month's  index  and  if 
ACWP (CUM)  -  ACWP ( CUM- 1)  »  o,  the  denominator  in  the  current 
CPI  will  equal  zero  and  any  further  calculations  which  use 
the  current  CPI  as  a  divisor,  such  as  EAC1,  will  yield  an 
undefinable  answer  as  a  result  of  division  by  zero.  Of  the 
819  data  points  in  the  database,  S  data  points  resulted 
in  division  by  zero.  These  data  points  were  treated  as 
missing  values  and  excluded  from  this  analysis.  The 
remaining  EAC  techniques  and  their  application  were  fully 
detailed  in  the  previous  chapter. 

Assumptions  Hade  When  Generating  EACs.  In  order  to 
test  which  EAC  formula  in  the  CPRA  program  is  the  better 
predictor  of  actual  cost  at  completion,  it  would  have  been 
ideal  to  analyze  data  from  completed  programs  (12).  However, 
data  for  completed  programs,  necessary  for  retroactively 
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evaluating  EACs,  is  not  maintained  in  the  CP;  A  database. 

Since  the  database  was  comprised  of  on-going  programs  in 
various  stages  of  completion,  it  was  necessary  to  arbitrarily 
select  a  point  which  could  simulate  the  actual  cost  at 
completion.  The  last  available  ACWP  measurement  for  each 
program  [ACWP (CUM)]  was  chosen  to  represent  this  cost  at 
completion  point  (12).  Effectives,  this  creates  a  program 
within  a  program  by  using  the  program  data  to__date  as  if  it 
were  data  from  a  completed  program.  This  makes  it  possible 
to  calculate  EACs  based  upon  this  subset  of  data  and  to 
evaluate  the  capability  of  each  EAC  technique  to  forecast 
the  actual  cost  f  work  performed  to  date.  This  value,  the 
the  last  reported  ACWP,  will  be  referred  to  as  the  Estimated 
Cost  at  Completion  (ECAC).  Because  ACWP (CUM)  has  been 
chosen  to  represent  the  actual  cost  at  completion,  it  is 
also  necessary,  for  calculation  purposes,  to  choose  a  data 
value  to  simulate  Budget  at  Completion  (BAC).  The  latest 
reported  BCWP(CUM)  is  the  logical  choice  since  it  represents 
the  budgetary  value  for  the  same  amount  of  work  for  which  the 
latest  ACWP ( CUM )  is  reported  (12). 

To  illustrate  the  above  methodology,  assume  an  on-going 
program  had  the  following  performance  measurements: 

Original  Target  Cost  =  200 

Current  Target  Cost  *  205 


The  estimated  cost  at  completion  (ECAC)  would  be  85  (the 
latest  ACWP).  The  budget  at  completion  (BAC)  would  be 
assumed  to  be  70  (because  this  is  the  budget  goal  for  the 
work  performed  to  date,  BCWP).  The  remaining  performance 
measurement  values  would  not  be  affected  by  these 
assumptions.  These  assumptions  provide  the  performance 
measurement  data  necessary  to  calculate  EACs  using  the  CPRA 
techniques . 

Statistical  Methodology 

The  problem  faced  by  the  researcher  is  one  of  choosing 
the  proper  statistical  technique  "to  describe  the  extent, 
direction,  and  strength  of  the  relationship  between  several 
independent  variables  and  a  continuous  dependent  variable" 
(16:11).  Regression  analysis*  is  well  suited  for  this 
purpose  (16:11)  and  has  been  selected  as  the  analytical  tool 
for  this  thesis.  This  section  describes  the  "statistical 
problems  of  finding  the  curve  (e.g.,  straight  line,  parabola, 
etc.)  that  best  fits  the  data  in  such  a  way  as  to  closely 
approximate  the  true  (but  unknown)  relationship  between  X 
and  Y"  (16:37, . 

There  are  several  strategies  for  studying  the 
relationship  between  two  variables  by  means 
of  regression  analysis.  The  most  common  of 
these  is  called  the  forward  method.  This 
strategy  begins  with  a  simply  structured  model. 


The  reader  who  is  not  familiar  with  regression  techniques 
may  wish  to  consult  Devore's  Probability  and  Statistics 
for  Engineering  and  the  Sciences  (14)  or  Neter  and 
Vasserman's  Applied  Linear  Statistical  Models  (17). 
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usually  a  straight  line  and  then  adds  more 
complexity  to  the  model  in  successive  steps,  if 
necessary  (16:38). 

Kleinman  and  Kupper  define  several  variations  of  this 
strategy,  but  deem  the  forward  method  of  regression  "the  most 
reasonable  strategy  to  use  in  the  absence  of  experience  or 
theory  to  indicate  otherwise'*  (16:39). 

Regression  Strategy.  A  univariate*  linear  regression 
was  used  to  formulate  a  regression  line  based  on  the  ECAC 
(the  dependent  variable  or  DV)  and  the  EAC  (the  independent 
variable  or  IV).  The  regression  equation  that  exhibited  the 
highest  sample  coefficient  of  correlation  was  defined  as  the 
"best  estimator"  of  the  cost  at  completion  (i.e.  the  better 
EAC  formula). 

Assumptions  for  Regression.  Certain  assumptions 
must  be  made  to  satisfy  the  straight  line  assumption  made 
when  using  linear  regression  (16:39).  These  assumptions 
are: 

1.  For  any  fixed  value  of  X,  Y  is  a  random  variable 
with  a  certain  probability  distribution  (16:41). 

2.  The  mean  value  of  Y  .  .  .  is  a  straight 
line  function  of  X  (16:42-44). 

3.  The  variance  [o3]  of  Y  is  the  same  for  any 
X.  This  assumption  is  called  the  assumption 
of  homoscedasticity  (16:44). 


"The  term  u  .a. variate  statistics  (univariate  regression 
in  this  case]  typically  refers  to  analyses  in  which  there  is 
a  single  DV"  (6:2). 

**  The  roots  of  the  term  "homoscedasticity"  are  "homo", 
meaning  "the  same"  and  "scedastic"  meaning  "scatter"  (16:44). 


4.  For  any  fixed  value  of  X,  Y  has  a  normal 
distribution  (16:44). 


Testing  the  Assumptions.  The  tests  of  the 
assumptions  for  regression  will  be  examined  and  explained  in 
this  section.  "Examination  of  the  residuals  scatterplots 
provides  a  test  of  the  assumptions  of  normality,  linearity, 
and  the  homoscedasticity  between  predicted  DV  scores  and 
errors  of  prediction"  (6; 93).  A  sample  size  greater  than  30 
is  usually  considered  to  be  of  adequate  dimension  to  invoice 
the  Central  Limit  Theorem  and  to  assume  approximate  normality 
(14:201).  Therefore,  the  815  data  records  in  the  CPRA 
database  are  assumed  to  have  a  normal  distribution. 

Linearity  of  relationship  between  predicted  DV  scores 
and  the  errors  of  prediction  is  also  assumed  (6:94).  If 
nonlinearity  is  present,  the  overall  shape  of  the  [residuals] 
scatterplot  will  be  curved  instead  of  rectangular"  (6:94-95). 
Examination  of  the  scatterplots  for  each  regression  did  not 
reveal  a  curvilinear  pattern  among  the  residuals.  Therefore, 
linearity  has  been  assumed. 

The  assumption  of  homoscedasticity  can  be  tested 
by  examining  the  width  of  the  band  in  which  most  of  the 
residuals  fall  (6:95).  If  the  width  of  this  band  is 
relatively  constant,  the  residuals  are  homoscedastic ; 
otherwise  the  residuals  are  heteroscedastic.  "When 
heteroscedasticity  prevails  but  the  other  conditions  of  the 
model  are  met,  the  estimators  bQ  and  b^  obtained  by 
ordinary  least  squares  procedures  are  still  unbiased  and 


consistent,  but  they  are  no  longer  minimum  variance  unbiased 
estimators"  (17:131).  This  data  did  exhibit  heteroscedastic 
tendencies  (i.e.  the  band  width  was  not  relatively  constant), 
however,  the  procedures  are  robust  for  these  violations. 

"Although  the  assumptions  are  important  in  derivation 
of  the  statistics,  they  are  frequently  less  important  in 
application  of  them  to  a  data  set"  (6:77).  "Univariate  tests 
of  significance  are  reasonably  robust  with  respect  to 
violations  of  assumptions  (e.g.  the  F  test  is  robust  to 
violations  of  normality  and  homogeneity  of  variance,  as  long 
as  sample  sizes  are  relatively  equal,  but  not  to  skewness)" 
(6:77).  Unfortunately,  the  extent  of  the  robustness  of 
various  test  to  various  violations  is  not  currently  known 
(6:77).  Probably,  however,  the  researcher  need  only  be 
concerned  about  flagrant  violations  of  the  assumptions 
(6:77).  While  the  data  did  exhibit  heteroscedastic 
tendencies  they  did  not  severely  degrade  the  analysis. 

The  General  Linear  Test. 

once  the  regression  lines  were  developed,  it  was 
possible  to  perform  a  aeries  of  general  linear  tests  ( GLTs ) 
among  them  to  determine  if  any  of  the  fitted  regression  lines 
were  statistically  equal.  As  hypothesized  in  Chapter  1,  the 
equality  of  two  (or  more)  regression  lines  would  indicate 
that  the  two  different  techniques  used  to  generate  EACs 
were  in  fact  different  expressions  of  the  same  EAC. 
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The  assumptions  for  the  general  linear  test  include 
those  assumptions  already  made  for  regression  and  the 
additional  assumption  of  the  equality  of  variance  (oj  =  03) 
between  test  groups.  An  F  test  was  used  to  check  the 
assumption  of  homogeneity  for  the  sample  variances  (17:165). 
The  hypotheses  are: 

cl*  °i  a  °2 

C2:  oj  *  o2 

The  test  statistic  is  then  calculated  as: 

F*  =  ((SSEx  +  (ni  -  2))  /  (SSE2  +  (n2  -  2))] 

The  critical  F  value  is: 

^(<x,  n^  -  2,  n2  *■  2) 

where 

a  =  the  probability  level 

ni  =  the  number  of  cases  in  data  group  1 

n2  =  the  number  of  cases  in  data  group  2 
This  resulted  in  an  Feritle4l  of  F(-05>  =1.0  As 

noted  earlier  "the  F  test  is  robust  to  .  .  .  violations  of 
homogeneity  of  variance  as  long  as  the  sample  sites  are 
relatively  equal"  (6:77).  The  results  of  these  tests 
indicated  that  the  variances  of  the  test  groups  were  equal  or 
that  the  test  was  robust  because  of  relatively  equal  sample 
sizes  in  all  but  one  case.  Theoretically ,  a  transformation 
of  the  data  to  stabilize  the  variances  between  the  two 
samples  could  have  been  performed,  however,  the  QLT  F* 
statistic  was  so  strong  that  a  transformation  was  not 
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warranted  (6:77).  The  results  of  this  test  are  given 
in  tabular  form  in  the  Results  chapter  of  this  thesis. 

General  Linear  Test  Procedure.  The  data  which 
generated  the.  parameters  (3q  and  for  each  of  the  two 

test  lines  was  combined  into  a  common  data  base  and  a  new 
regression  equation  calculated.  Next,  the  error  portion  of 
the  new  line,  referred  to  as  the  reduced  model,  was  compared 
to  the  combined  error  of  the  test  lines  and  referred  to  as 
the  full  model.  The  term  SSE(R)  is  used  to  describe  the 
sum  of  the  squared  error  for  the  reduced  model.  The  terms 
SSE^  and  SSE2  describe  the  sum  of  the  squared  error  for 
regression  lines  1  and  2  respectively.  Neter  and  Wasserman 
describe  the  steps  performed  in  comparing  the  respective  SSEs 
below  (17:161). 

1.  Fit  the  full,  or  unrestricted,  model  and 
obtain  the  error  sum  of  the  squares  SSE(F). 

The  SSE  for  the  full  model  is 

SSE(F)  =  SSEx  +  SSE2  (17:161). 

2.  Obtain  the  reduced,  or  restricted,  model 
under  ,  fit  it,  and  determine  the  error  sum  of 
squares  SSE(R)  for  the  reduced  model  (17:161). 

3.  Calculate  the  F*  statistic  .  .  .  which 
involves  the  difference  SSE(R)  -  SSE(F).  The 
greater  the  difference,  the  more  the  data  support 
(hypothesis]  C2>  the  smaller  the  difference,  the 
more  the  data  support  [hypothesis]  (17:161). 


Ci  and  C 2  have  been  used  to  permit  the  reader  to  follow  the 
lengthy  discussion  of  the  general  linear  test  in  Neter  and 
Wasserman' s  Applied  Linear  Statistical  Models  (17:160-165). 
Introductory  statistics  texts  such  as  Devore's  Probability  & 
Statistics  for  Engineering  and  the  Sciences  (14)  use  the  more 
common  terminology  of  H0  for  the  null  hypothesis  and  H4  for 
the  alternative  hypothesis. 
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where  and  C2  are  the  hypotheses, 

cl:  hi  ~  ^02  and  ^11  "  ^12 

! 

C2:  Either  {JQ1  ^  (3q2  or  ?  P2I  or 

If  the  two  regression  lines  are  the  same  (C^), 
both  the  intercept  and  the  slope  terms  must  be 
equal.  If  the  regression  lines  are  not  the  same 
|  (C2),  they  must  differ  with  either  respect  to  the 

intercept  or  the  slope  or  with  respect  to  both 
(17:163). 

The  test  statistic  used  to  evaluate  the  hypotheses 
was  the  F  statistic.  The  components  of  the  test  statistic 
are  shown  below. 

Fn  =  _ SSE(R)  -  SSE(F) _ 

(n^  +  n2  “  2)  -  (n^  +  n2  *  4) 

Fd  =  SSE(F) 
nl  +  n2  -  4 

Next,  the  test  statistic  F*  was  calculated 
F*  *  fn  +  FD 

Finally,  the  critical  value  of  F  was  obtained  in  the  form 

F/i  „  . .  where 

(1  -  a;  2,  n^  +  n2  -  4) 

a  =  the  probability  level 
r\i  =  the  number  of  cases  in  data  group  1 
n2  =  the  number  of  cases  in  data  group  2 
The  decision  rule  for  controlling  the  risk  of  a  Type  1 
error  at  a  is  (17:165): 

I£  F*  5  F(1  -  a,  2,  m  +  n2  -  4)*  concise  cl* 
the  regression  lines  being  tested  are  equal, 
or 

I£  F*  >  F(1  -  a,  2,  nx  +  n2  -  41-  oonclude  c2< 
the  regression  lines  being  tested  are  not  equal. 
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The  a,  or  level  of  significance,  used  for  the  statistical 
tests  conducted  for  this  thesis  was  a  =  .05 
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III.  Results 


This  chapter  presents  the  results  of  the  statistical 
procedures  described  in  the  Methodology  chapter  of  this 
thesis.  First,  the  results  of  the  regression  of  the 
estimated  cost  at  completion  (ECAC)  with  each  CPRA  EAC 
technique  are  summarized  in  Table  3.1.  Second,  the  results 
of  the  intermediate  steps  necessary  to  conduct  the  general 
linear  test  have  been  summarized  and  displayed.  These  steps 
consist  of  the  test  for  homogeneity  (equality)  of  variance 
(Table  3.2)  among  the  samples  and  the  regression  of  the  ECAC 
with  any  two  CPRA  EACs  (Table  3.3).  Finally,  the  results  of 
the  general  linear  test  for  equality  among  the  EAC  regression 
lines  have  been  arranged  in  tabular  form  and  are  displayed  in 
Table  3.4. 

Linear  Regression  Results  for  EACs.  The  following 
results  were  generated  with  SPSS,  the  computer  based 
statistical  analysis  program  described  earlier.  The  results 
shown  in  Table  3.1  are  a  summary  of  the  results  obtained 
by  regressing  each  EAC  technique  and  the  estimated  actual 
cost  at  completion.  The  results  of  each  individual 
regression  may  be  found  in  Appendix  A. 


TABLE  3.1 


EAC  Regression  Summary 


R  SQUARE 

SSE 

EACl 

.16509 

.97163E+15 

EAC  2 

.93771 

.73215E+14 

EAC  3 

.89534 

.10589E+15 

EAC4 

.80030 

.15436E+15 

EAC  5 

.93801 

.72680E+14 

EAC  6 

.73172 

.  31338E+15 

The  R  SQUARE  shown  in  this  table  is  the  coefficient  of 
determination.  Since  the  R  SQUARE  value  is  the  ratio  of 
explained  variance  to  total  variance,  the  EAC  with  the 
highest  R  SQUARE  value  explains  the  most  variance  and  has 
previously  been  defined  as  the  better  predictor. 

General  Linear  Test  Data.  The  results  of  the  test  for 
homogeneity  of  variance  and  the  application  of  linear 
regression  to  the  combinations  of  data  required  to  perform 
the  general  linear  tests  described  in  the  methodology  chapter 
are  summarized  in  the  tables  below.  The  results  of  the  test 
for  homogeneity  of  variance  are  summarized  in  Table  3.2. 


TABLE  3.2 


Test  Results  for  Homogeneity  of  Variance 


TEST 

EACS 

*2 

_* 

F 

EACl 

AND 

EAC2 

8.1E+02 

8.1E+02 

13.40 

EACl 

AND 

EAC3 

8.1E+02 

7.0E+02 

7.98 

EACl 

AND 

EAC4 

.8. 1E+02 

5.4E+02 

4.18 

EACl 

AND 

EAC5 

8.1E+02 

8.1E+02 

13.47 

EACl 

AND 

EAC6 

8.1E+02 

8.1E+02 

3.11 

EAC2 

AND 

EAC3 

8 . 1E+02 

7.0E+02 

0.60 

EAC2 

AND 

EAC4 

8.1E+02 

5.4E+02 

0.31 

EAC2 

AND 

EAC5 

8.1E+02 

8.1E+02 

1.00 

EAC2 

AND 

EAC6 

8.1E+02 

8.1E+02 

0.23 

EAC3 

AND 

EAC4 

7.0E+02 

5.4E+02 

0.52 

EAC3 

AND 

EAC5 

7.0E+02 

8.1E+02 

1.69 

EAC3 

AND 

EAC6 

7.0E+Q2 

8.1E+02 

0.39 

EAC4 

AND 

EAC5 

S.4E+02 

8.1E+02 

3.22 

EAC4 

AND 

EAC6 

5.4E+02 

8.1E+02 

0.74 

EAC5 

AND 

EAC6 

8.1E+02 

8.1E+02 

0.23 

The  'critical  of  f(.05,  «(  «)  s  1-°  had  t0  **  greater  than 
the  F*  value  or  the  sample  sizes  had  to  be  relatively 

equal  to  permit  assumption  of  equality  of  variance  among  the 
samples.  Table  3.2  indicates  at  least  one  of  these 
conditions  was  met  in  all  but  one  case.  That  case,  EACl  and 
EAC4,  was  discussed  in  the  methodology  chapter  on  general 
linear  test  procedures. 
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The  regression  data  summarized  in  Table  3.3  is  fully 
detailed  in  Appendix  B.  The  SSE  shown  is  the  SSE(R)  used 
when  calculating  the  GLT  F*  test  statistic.  These 
intermediate  calculations  were  necessary  to  derive  the  SSE(R) 
values,  without  which,  the  general  linear  test  could  not  have 
been  performed. 

TABLE  3.3 

Summary  for  General  Linear  Test  Regressions 


R  SQUARE 

SSE 

EACl  AND  EAC2 

.23580 

.17875E+16 

EAC1  AND  EAC3 

.22661 

.16826E+16 

EACl  AND  EAC4 

.20918 

. 15318E+16 

EACl  AND  EAC5 

.23192 

.17944E+16 

EACl  AND  EAC6 

.24130 

. 17692E+16 

EAC2  AND  EAC3 

.91693 

. 18169E+15 

EAC2  AND  EAC4 

.87683 

.  24000E+15 

EAC2  AND  EACS 

.93761 

.  14647EHS 

EAC2  AND  EAC6 

.81252 

.43936E+1S 

EAC3  AND  EAC4 

.85198 

. 26423E+15 

EAC3  AND  EACS 

.91543 

.18473E+15  1 

EAC  AND  EAC6 

. 7926S 

.452Q2E+15 

EAC4  AND  FACS 

>87304 

.24703E+1S 

EAC4  AND  EAC6 

.75312 

. 47926E+15 

EACS  AND  EAC6 

.80890 

.44728E+15 
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General  Linear  Test  Results*  Table  3.4  indicates  the 
EACs  which  were  tested,  the  F*  and  Fcritical  (Fc)  values. 
The  formula  used  to  derive  the  F*  test  statistic  and  the 
Fc  may  be  found  in  the  methodology  chapter  of  this  thesis. 
Since  F*  was  greater  than  Fc  in  all  cases,  all  of  the 
regression  lines  may  considered  unique. 


TABLE  3.4 

General  Linear  Test  Results 


TEST 

EACS 

F* 

EAC1 

AND 

EAC  3 

421.44 

3.00 

EAC1 

AND 

EAC  4 

240.75 

3.00 

EAC1 

AND 

EAC5 

578.92 

3.00 

EAC1 

AND 

EAC6 

303.13 

3.00 

EAC2 

AND 

EAC3 

10.89 

3.00 

EAC  2 

AND 

EAC4 

3S.69 

3.00 

EAC2 

AND 

EACS 

3.19 

3.00 

EAC2 

AND 

EACS 

110.35 

3.00 

EAC3 

AND 

SAC4 

9.41 

3.00 

EAC3 

AND 

EACS 

2S.99 

3.00 

EAC  3 

AND 

EAC6 

58.74 

3.00 

EAC4 

AND 

EACS 

59.08 

3.00 

EAC4 

AND 

SACS 

16.49 

3.00 

EACS 

AND 

EACS 

128.05 

3.00 

in  other  words,  this  data  indicates  that  there  no  two  EAC 
formulas  yield  the  same  EAC. 


IV.  Discussion  and  Conclusion. 


A  Review  of  the  Hypothesis. 

The  research  hypothesis  stated  that  a  statistical 
evaluation  of  the  past  performance  of  each  EAC  method  would 
indicate  whether  a  ''better'*  method  of  formulation  existed 
and,  if  soothe  evaluation  would  identify  that  method.  This 
hypothesis  was  tested  by  calculating  EACs  using  each  of  the 
CPRA  techniques  and  regressing  them  with  the  ECAC  for  each 
program. 

Conclusion. 

The  regression  model  for  EACS  explained  the  highest 
percentage  of  variation  and  has  been  selected  as  the  "best 
predictor".  The  CPRA  EAC  techniques  are  ranked  from  the 
highest  to  lowest  percentage  of  explained  variation  (R 
SQUARE)  in  table  4.1. 

TABLE  4.1 
EAC  Rankings 


EAC  NAME  R 

SQUARE 

SSE 

EAC  5 

-  WEIGHTED  Cri/SPI 

.93801 

. 72680E+14 

EAC  2 

-  CUMULATIVE  CPI 

.93771 

.73215E+14 

EAC  3 

-  3  MON  CPI 

.89534 

.10589E+15 

EAC  4 

-  ACWP  REGRESSION 

.80030 

. 15436E+15 

EAC6 

-  TREND  WEIGHTED  CPI/SPI 

.73172 

. 31338E+15 

EAC1 

-  MONTHLY  CPI 

.16509 

.97163E+15 
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Discussion 


It  is  clear  from  Table  4.1  that  the  WEIGHTED  CPI/SPI 
and  the  CUMULATIVE  CPI  are  closely  matched  in  their 
respective  predictive  power.  The  remainder  of  the  techniques 
did  provide  good  (as  measured  by  R  SQUARE)  predictive 

i 

capability  and  deserve  the  attention  of  the  analyst.  EAC1, 
the  MONTHLY  CPI  technique,  was  the  only  poor  predictor  of  the 
six  techniques  tested.  This  may  be  due  to  the  fact  that  any 
variation  in  monthly  performance  is  completely  projected  into 
cost  of  the  remaining  work.  For  example,  if  one  month’s  data 
indicated  the  the  contractor  was  performing  at  a  50% 
efficiency  level,  EAC1  assumes  the  remaining  work  effort  will 
be  accomplished  ac  that  same  50%  level  of  efficiency.  Of 
course,  this  tends  to  result  in  significant  variations  from 
BAC. 

The  results  of  the  General  Linear  Test  for  equality 
among  the  regression  lines  indicate  that  there  was  no 
redundancy  among  the  EAC  techniques  used  in  the  CPRA  program. 

Recommendations .  Since  these  conclusions  were  based  on 
the  application  cf  EAC  calculation  techniques  on  performance 
measurement  data  for  on-going  research  and  development 
programs,  it  may  be  appropriate  to  question  their 
applicability  to  either  full  scale  development  or  production 
programs.  This  issue  is  recognised,  but  has  not  been 
addressed  in  this  thesis.  This  analysis  has  provided  a 
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simple  decision  rule  for  the  selection  of  EAC  techniques. 
Future  research  could  analyze  this  data  in  terms  of  total 
contract  cost,  acquisition  phase  of  the  program,  type  of 
contract,  or  the  percentage  of  the  contract  completed. 
Research  in  these  area's  would  provide  the  field  analyst  a 
more  precise  guide  to  EAC  technique  application  for  a  given 
set  of  conditions. 


Appendix  A:  Regression  Results 
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TABLE  A.l 

Regression  Results  for 

EACl 

MEAN 

STD  DEV 

CASES 

ECAC 

827534.056 

1203107.217 

815 

EACl 

1017017.360 

3875706.202 

805 

MULTIPLE  R  .40632 

R  SQUARE  .16509  R  SQUARE  CHANGE  .16509 
ADJUSTED  R  SQUARE  .16405  F  CHANGE  158.78393 
STANDARD  ERROR  .11000E+07  SIGNIF  F  CHANGE  .0000 


ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES 
REGRESSION  1  .19213E+15 
RESIDUAL  803  .97163E+1S 
F  =  158.78393  SIGNIF  F  = 


OF  SQUARES  MEAN  SQUARE 
. 19213E+15  .19213E+15 
.97163E+15  .12100E+13 
SIGNIF  F  =  .0000 


TABLE  A. 2 


Regression  Results  for  EAC2 


TABLE  A. 3 


Regression  Results  for  EAC3 


MEAN 

STD  DEV  CASES 

ECAC  827534.056 

1203107.217  815 

EAC3  815483.312 

1281310.244  700 

MULTIPLE  R 

.94622 

R  SQUARE 

.89534  R  SQUARE  CHANGE  .89534 

ADJUSTED  R  SQUARE 

.89519  F  CHANGE 

§971,27998 

STANDARD  ERROR  389496.39744  SIGNIF  F 

CHANGE  0 

ANALYSIS  OF  VARIANCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.90589E+15 

.90569E+15 

RESIDUAL  698 

. 10589E+15 

. 15171E+12 

F  a  5971.27998 

SIGNIF  F  « 

0 

TABLE  A. 4 


Regression  Results  for  EAC4 


MEAN  STD  DEV  CASES 
ECAC  827534.0561203107.217  815 
EAC4  737831.309  1283567.883  535 

MULTIPLE  R  ,89459 

R  SQUARE  .80030  R  SQUARE  CHANGE  .80030 
ADJUSTED  R  SQUARE  .79992  F  CHANGE  2135.98328 
STANDARD  ERROR  538148.37577  SIGNIF  F  CHANGE  .0000 

ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES  MEA.J  SQUARE 
REGRESSION  1  .6i8S9Evl5  ;  .61859E+15 
RESIDUAL  533  .15436E+15  .28960E+12 
F  = 


2135.98328 


SIGNIF  F  « 


C000 


TABLE  A. 5 


Regression  Results  for  EAC5 


MEAN 

STD  DEV  CASES 

ECAC  827534.056 

1203107.217  815 

EAC5  800919.116 

1232852.068  811 

MULTIPLE  R 

.96851 

R  SQUARE 

.93801  R  SQUARE 

CHANGE  .93801 

ADJUSTED  R  SQUARE 

.93793  F  CHANGE 

12241.41679 

STANDARD  ERROR  299733.06199  SIGNIF  F  CHANGE  .0000 

ANALYSIS  OF  VARIANCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.10998E+16 

.10998E+16 

RESIDUAL  809 

.72680E+14 

. 89840E+11 

F  =  12241.41679 

SIGNIF  F  =  .0000 

TABLE  A. 6 

Regression  Results  for  EAC6 


MEAN 

STD  DEV  CASES 

ECAC  827534.056 

1203107.217  815 

EAC6  898976.353 

1546542.481  808 

MULTIPLE  R 

.85541 

R  SQUARE 

.73172  R  SQUARE 

CHANGE  .73172 

ADJUSTED  R  SQUARE 

.73139  F  CHANGE 

2198.34905 

STANDARD  ERROR  623542.27794  SIGNIF  F 

CHANGE  0 

ANALYSIS  OF  VARIANCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.85473E+15 

.85473E+15 

RESIDUAL  806 

.31338E+15 

.  38880E+12 

F  =  2198.34905 

SIGNIF  F  - 

0 

TABLE  B.l 

General  Linear  Test  for  EAC1  and  EAC2 

MULTIPLE  R  .48559 

R  SQUARE  .23580 

ADJUSTED  R  SQUARE  .23533 

STANDARD. ERROR  . 10517E+07 

ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES  MEAN  SQUARE 

REGRESSION  1  .55157E+15  .55157E+15 

RESIDUAL  1616  .17875E+16  .11062E+13 

F  = 


R  SQUARE  CHANGE  .23580 
F  CHANGE  498.63505 
SIGNXF  F  CHANGE  .0000 


498.63505 


SIGNXF  F  =  .0000 


TABLE  B.2 


General  Linear  Test  for  EACl  and  EAC3 


MEAN 

STD  DEV  CASES 

ECAC 

827534.056 

1202737.882  1630 

EACS 

923280.594 

2966990.159  1505 

MULTIPLE 

R 

.47603 

R  SQUARE 

.22661  R  SQUARE 

CHANGE 

.22661 

ADJUSTED 

R  SQUARE 

.22609  F  CHANGE 

440.38882 

STANDARD 

ERROR  . 10581E+07  SIGNIF  F 

CHANGE 

0 

ANALYSIS  OF  VARIANCE 
DF 

REGRESSION  1 
RESIDUAL  ISO 3 
F  = 


SUM  OF  SQUARES 
.49302E+15 
. 16826E+16 


MEAN  SQUARE 
.49302E+15 
. 11195E+13 


440.38882 


SIGNIF  F  * 


0 


Table  B.3 


General  Linear  Test  for  EAC1  and  EAC4 


MEAN  STD  DEV  CASES 
ECAC  827534.056  1202737.882  1630 
EACS  905551.287  3113703.421  1340 

MULTIPLE  R  .45736 
R  SQUARE  .20918 
ADJUSTED  R  SQUARE  .20859 
STANDARD  ERROR  .10700E+0? 

ANALYSIS  OF  VARIANCE 


DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

. 40517E+15 

.4G517E+15 

RESIDUAL  1338 

. 15318E+16 

. 11448E+13 

F  a  353.90767 

SIGNIF  F  a 

.0000 

R  SQUARE  CHANGE  .20918 
F  CHANGE  353.90767 
SXGNIF  F  CHANGE  .0000 


TABLE  B.4 


General  Linear  Test  for  EAC1  and  EAC5 


MEAN  STD  DEV  CASES 
827534. 05S  1202737.882  1630 
903567.065  2872628.141  1616 


ECAC 
EACS 

MULTIPLE  R  .48158 
R  SQUARE  .23192 
ADJUSTED  R  SQUARE  .23144 
STANDARD  ERROR  .10544E+07 


R  SQUARE  CHANGE  .23192 
F  CHANGE  487.34045 
SIGNXF  F  CHANGE  .0000 


ANALYSIS  0?  VARIANCE 


DF 

REGRESSION  1 
RESIDUAL  1614 

P  S3 


SUM  OF  SQUARES 
. 54161E+15 
.  17944E+16 
3IGNIF  F  « 


MEAN  SQUARE 
.54181E+15 
. 11118E+13 

0000 


487.34045 


TABLE  B.S 


General  Linear  Test  for  EAC1  and  EAC6 

MEAN  STD  DEV  CASES 
ECAC  827534.056  1202737.882  1630 
EACS  957887.085  2948351.882  1613 

MULTIPLE  R  .49122 

R  SQUARE  .24130  R  SQUARE  CHANGE  .24130 
ADJUSTED  R  SQUARE  .24083  P  CHANGE  512.36274 
STANDARD  ERROR  .1048QE-M37  SXGNIF  F  CHANGE  0 

ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES  MEAN  SQUARE 
REGRESSION  1  .56268E+15  .56268BH5 
RESIDUAL  1611  .17692E+16  .10982E+13 
F  *  512.36274  SIGNIF  F  »  0 


"'.''Vi 
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TABLE  B.6 

General  Linear  Test  for  EAC2  and  EAC3 


MEAN 

STD 

DEV  CASES 

ECAC  827534.056 

1202737. 

882  1630 

EACS  822857.425 

1270337. 

714  1513 

MULTIPLE  R 

.95757 

R  SQUARE 

.91693 

R  SQUARE 

CHANGE  .91693 

ADJUSTED  R  SQUARE 

.91688 

F  CHANGE 

16678.65063 

STANDARD  ERROR  346764. 5611S 

SIGNIF  F 

CHANGE  .0000 

ANALYSIS  OF  VARIANCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.20055E+1S 

.20055E+16 

RESIDUAL  1511 

. 18169E+15 

. 12025E+12 

F  =  16678.65063 

SIGNIF  F  -  .0000 

a 


f  } 
t 


;v£\t* 


TABLE  B.7 


General  Linear  Test  for  EAC2  and  EAC4 


MEAN  STD  DEV  CALEB 
ECAC  827534.056  1202737.882  1630 
EACS  792941.183  1270657.850  1348 

MULTIPLE  R  .93639 
R  SQUARE  .87683 
ADJUSTED  R  SQUARE  ,87674 
STANDARD  ERROR  422264.01448 

ANALYSIS  OP  VARIANCE 

DF  SUM  OF  SQUARES  MEAN  SQUARE 
REGRESSION  1  .17085E+16  .17085E+16 
RESIDUAL  1346  .24000E+15  .17831E+12 
F  s 


R  SQUARE  CHANGE  .87683 
F  CHANGE  9582.01875 
SIGNIF  F  CHANGE  .0000 


9502.01875 


SIGNIF  F  a  .0000 


TABLE  B. 8 


General  Linear  Test  for  EAC2  and  EAC5 


MEAN 

STD  DEV  CASES 

ECAC 

827534.056 

1202737.882  1630 

EACS 

815080.275 

1247006.818  1624 

MULTIPLE 

R 

.96830 

R  SQUARE 

.93761  R  SQUARE  CHANGE  .93761 

ADJUSTED 

R  SQUARE 

.93757  F  CHANGE 

24377.09413 

STANDARD 

ERROR  300504.61205  SIGNIF  F 

CHANGE  0 

ANALYSIS 

OF  VARIANCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.22013E+16 

.22013E+16 

RESIDUAL 

1622 

.  14647E+15 

.90303E+11 

F  =  24377.09413 

SIGNIF  F  a 

0 

TABLE  B. 9 


General  Linear  Test  for  EAC2  and  EAC6 

MEAN  STD  DEV  CASES 

ECAC  827534.056  1202737.882  1630 

EACS  863983.873  1410825.398  1621 

R  SQUARE  CHANGE  .81252 
F  CHANGE  7016.36954 

SIGNIF  F  CHANGE  0 

ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES  MEAN  SQUARE 

REGRESSION  1  .19041E+16  .19041E+16 

RESIDUAL  1619  .43936E+15  .27138E+12 

F  = 


MULTIPLE  R  .90140 
R  SQUARE  .81252 
ADJUSTED  R  SQUARE  .81240 
STANDARD  ERROR  520940.39336 


7016.36954 


SIGNIF  F 


0 


TABLE  B.10 

General  Linear  Test  for  EAC3  and  EAC4 


MEAN 

STD  DEV  CASES 

ECAC  827534.056 

1202737.882  1630 

EACS  781844.590 

1281346.701  1235 

MULTIPLE  R 

.92303 

R  SQUARE 

.85198  R  SQUARE 

CHANGE  .85198 

ADJUSTED  R  SQUARE 

.85186  F  CHANGE 

7096.85440 

STANDARD  ERROR  462924.17884  SXGNIF  F 

CHANGE  0 

ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.1S208E+16 

.15208E+16 

RESIDUAL  1233 

. 26423E+15 

. 21430E+12 

F  -  7096.85440 

SXGNIF  F  a 

0  1 

TABLE  B. 12 


General  Linear  Test  for  EAC3  and  EAC6 

MEAN  STD  DEV  CASES 
ECAC  827534.056  1202737.882  1630 
EACS  860219.637  1429701.872  1508 

MULTIPLE  R  .89031 
R  SQUARE  .79265 
ADJUSTED  R  SQUARE  .79251 
STANDARD  ERROR  547855.53339 

ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES  MEAN  SQUARE 
REGRESSION  1  .17280E+16  .17280E+16 
RESIDUAL  1506  .45202E+15 


R  SQUARE  CHANGE  .79265 
F  CHANGE  5757.11846 
SIGNIF  F  CHANGE  .0000 


30015E+12 


TABLE  B.13 

General  Linear  Test  for  EAC4  and  EAC5 


MEAN 

STD  DEV  CASES 

ECAC  827534.056 

1202737.882  1630 

EACS  775843.353 

1253162.968  1346 

MULTIPLE  R 

.93436 

R  SQUARE 

.87304  R  SQUARE  CHANGE  .87304 

ADJUSTED  R  SQUARE 

.87294  F  CHANGE 

9241.76050 

STANDARD  ERROR  428717.39021  SIGNIF  F 

CHANGE  0 

ANALYSIS  OF  VARIANCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.16986E+16 

. 16986E+16 

RESIDUAL  1344 

•24703E+15 

. 18380E+12 

F  =  9241.76050 

SIGNIF  F  * 

0 

TABLE  B. 14 


General  Linear  Test  for  EAC4  and  EAC6 

MEAN  STD  DEV  CASES 
ECAC  827534.056  1202737.882  1630 
EACS  834782.311  1449169.993  1343 

MULTIPLE  R  .86783 
R  SQUARE  .75312 
ADJUSTED  R  SQUARE  .75294 
STANDARD  ERROR  597822.50588 

ANALYSIS  OF  VARIANCE 

DF  SUM  OF  SQUARES  MEAN  SQUARE 
REGRESSION  1  .14620E+16  .14620E+16 
RESIDUAL  1341  .47926E+15  .3S739E+12 
F  »  4090.87759  SXGNIF  F  e  0 


R  SQUARE  CHANGE  .75312 
F  CHANGE  4090.87759 
SIGNIF  F  CHANGE  0 


TABLE  B.15 

General  Linear  Test  for  EAC5  and  EAC6 


MEAN 

STD  DEV  CASES 

• 

ECAC  827534.056 

1202737.882  1630 

EACS  849856.885 

1398659.043  1619 

MULTIPLE  R 

.89939 

R  SQUARE 

.80890  R  SQUARE 

CHANGE  .80890 

ADJUSTED  R  SQUARE 

. 80878  F  CHANGE 

6844.60388 

STANDARD  ERROR  525937.21652  SXGNXF  F 

CHANGE  0 

ANALYSIS  OF  VARIANCE 

DF 

SUM  OF  SQUARES 

MEAN  SQUARE 

REGRESSION  1 

.18933E+16 

.18933E+16 

RESIDUAL  1617 

.44728E+15 

. 27661E+12 

F  a  6844.60388 

SXGNXF  F  = 

0 
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